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ABSTRACT 


The  key  to  effective  military  personnel  planning  is  accurate  loss  forecasting. 
Accurate  estimates  of  future  losses  enable  personnel  managers  to  determine  the  number 
of  individuals  to  recruit  and  promote,  as  well  as  the  size  and  cost  of  future  personnel 
inventories. 

The  thesis  describes  the  generation  and  analysis  of  several  simple  loss  rate 
forecasting  models.  The  models  are  divided  into  two  classes,  those  that  incorporate 
eligibility  data  and  those  that  do  not. 

Aviation  officers,  particularly  pilots,  were  narrowed  down  to  Lieutenants  with 
four  to  nine  years  of  commissioned  service.  They  were  divided  into  three  communities 
(jet,  prop,  and  helo). 

Two  methods  of  loss  forecasting  were  used,  131  which  is  somewhat  akin  to  OP-Ols 
technique  and  the  method  I  wish  to  exploit,  EAOS. 

EAOS  techniques  appear  to  contribute  more  to  officer  loss  forecasting  than  the 
BI  technique.  However,  BI  techniques  are  still  significant  but  to  a  lesser  degree.  The 
findings  are  discussed  within  the  context  of  the  study. 
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I.  INTRODUCTION 

A.  PROBLEM 

The  key  to  effective  military  personnel  planning  is  accurate  loss  forecasting. 
Accurate  estimates  of  future  losses  enable  personnel  managers  to  determine  the  number 
of  individuals  to  recruit  and  promote,  as  well  as  the  size  and  cost  of  future  personnel 
inventories.  Officer  personnel  losses  can  generally  be  grouped  into  two  distinct  types: 
voluntary  and  involuntary.  The  voluntary  portion  (e.g.,  resignations,  most  retirements) 
is  the  largest  and  most  volatile.  Time  series  and  econometric  methods  are  used 
currently  to  forecast  voluntary  loss  rates.  Both  classes  of  methods  focus  on  trends  in 
historical  loss  rates  (through  data  processing,  all  historical  losses  are  partitioned  into 
either  voluntary  or  involuntary  groups).  The  present  modelling  techniques  incorporate 
little  or  no  information  about  the  ability  of  a  current  or  future  population  of  officers  to 
make  a  voluntary  decision  to  stay  or  leave  the  Navy.  Put  differently,  the  techniques  do 
not  consider  the  portion  of  a  begin-year  inventory  that  is  at  risk  or  eligible  to  leave. 

The  Navy  Personnel  Research  and  Development  Center  (NPRDC)  has  developed 
Navy  officer  End-of-Active-Obligated  Service  (EAOS)  dates  for  all  officers  on  active 
duty  at  the  end  of  FYs  1974-87.  These  data  determine  who  is  eligible  to  leave  the  Navy 
immediately  (no  obligation),  who  will  be  free  of  obligation  sometime  in  the  upcoming 
fiscal  year  (less  than  one  year  remaining  on  contract),  and  who  will  be  ineligible  to 
leave  during  the  fiscal  year  (greater  than  one  year  remaining  on  contract).  When 
aggregated,  the  first  two  groups  comprise  the  population  at  risk  for  making  voluntary 
loss  decisions.  To  date,  these  data  have  not  been  exploited  in  any  of  the  loss  forecasting 
techniques.  It  is  felt  that  this  information  can  contribute  to  improved  forecast  accuracy 
of  existing  methods,  especially  by  anticipating  turning  points  in  loss  behavior.  This 
thesis  examines  quantitatively  the  usefulness  of  these  eligibility  data  to  accurate  officer 
loss  forecasting. 

B.  OBJECTIVE 

The  thesis  describes  the  generation  and  analysis  of  several  simple  loss  rate 
forecasting  models.  The  models  are  divided  into  two  classes,  those  that  incorporate 
eligibility  data  and  those  that  do  not.  The  study  compares  the  forecast  accuracy  of 
these  methods  over  a  five  year  historical  period  (FY1982- 1986).  The  validation  results 


are    presented   in   several  ways   and   analyzed   to    determine    if  the   eligibility   data 
contributes  to  improved  loss  forecasting. 

C.       PREVIOUS  STUDIES 

In  a  1978  report,  Edward  Bres  and  Murray  Rowe,  analysts  from  NPRDC, 
describe  a  study  to  improve  existing  Unrestricted  Line  (URL)  loss  rate  forecasting 
methods.   [Ref.  1] 

A  number  of  standard  extrapolation  techniques  (e.g.,  averaging,  exponential 
smoothing,  time  series)  were  tested  to  forecast  loss  rates  for  the  total  URL.  These 
techniques  used  historical  loss  rate  data  (FY  1969-1977)  arrayed  by  paygrade,  years  of 
commissioned  service  (YCS)  ,  and  promotion  status.  Estimates  were  compared  to  find 
the  most  accurate  forecasting  technique,  based  on  their  relative  predictive  accuracy  over 
time. 

A  technique  was  judged  superior  to  another  if  it  had  a  lower  mean  absolute  error 
(MAE).  The  analysis  showed  that  an  Autoregressive  Time  Series-Minimum  Absolute 
Deviation  (MAD--3  years)  technique  had  the  minimum  MAE  in  68  percent  grade/ YCS 
cells.  Alternatively,  the  method  used  by  OP-01  at  the  time,  Weighted  Moving  Average 
(7  years),  produced  the  best  results  in  only  10.7  percent  of  the  cells. 

In  October  of  1980,  Bres  and  Rowe  conducted  another  study  which  would 
improve  the  measure  of  Navy  officer  retention  [Ref.  2].  Currently  used  by  offices  of 
the  Deputy  Chief  of  Naval  Operations  (Manpower,  Personnel,  and  Training)  (OP-01) 
was  the  retention  rate  (RR).  The  RR  was  used  to  assess  the  Navy's  ability  to  build  and 
maintain  a  career  force  of  officers.  This  rate  employed  the  minimum  service 
requirement  (MSR)  as  a  point  of  reference2.  However,  since  the  MSR  frequently  did 
not  reflect  an  officer's  true  obligation,  the  RR  tended  to  be  overstated.  In  turn,  the 
maintainabilitv  of  the  career  force  was  overstated. 


^'ears  of  commissioned  service  are  computed  from  an  officer's  relative  year 
group.  This  year  group  is  the  fiscal  year  of  first  commissioning.  An  officer's  relative 
year  group  is  then  the  number  of  years  between  the  current  fiscal  year  and  that  first 
commissioning.  Hence,  relative  year  group  serves  as  a  measure  of  length  of 
(commissioned)  service. 

2Minimum  service  requirement  (MSR)  is  the  obligated  service  an  officer  incurs  as 
a  result  of  initial  commissioning  and  specialized  warfare  training.  The  MSR  differs  by 
commissioning  source  (e.g.,  a  Naval  Academy  graduate  receives  a  longer  MSR  than  an 
Officer  Candidate  School  graduate.)  and  by  warfare  training,  (e.g.,  the  service 
obligation  for  flight  training  exceeds  that  for  Surface  Warfare  Officers'  School). 


The  Base  Force  Retention  Rate  (BFR)  was  intended  to  replace  the  outdated  RR. 
The  BFR  did  not  rely  on  MSR-based  computations.  It  was  simply  the  proportion  of 
the  career  force  base  (CFB)  (officers  between  5  and  10  years  of  commissioned  service) 
who  resign  their  commissions  annually  . 

Bres  and  Rowe  found  that  the  BFR  was  most  useful  for  identifying  changes  or 
trends  in  the  overall  retention  behavior  of  a  community's  CFB--trends  that  the 
conventional  RR  might  not  reveal  at  all  or  might  uncover  much  later  than  would  the 
BFR. 

To  properly  address  the  feasibility  of  future  officer  manpower  needs  required  the 
simultaneous  consideration  of  manpower  requirements,  the  existing  and  projected 
personnel  inventory,  and  the  projected  supply  of  new  officers.  All  of  these  management 
functions  are  interrelated,  but  they  are  organizationally  distinct  and  lack  the  coherent 
linkages  necessary  to  respond  adequately  and  rapidly  to  planning  and  programming 
questions.  To  come  closer  to  the  way  the  officer  manpower  system  really  works  and  to 
improve  response  time,  officer  manpower  management  must  function  dynamically 
using  a  common  set  of  models  and  policies. 

In  1981,  NPRDC  deployed  the  initial  version  of  the  Structured  Accession 
Planning  System  for  Officers  (STRAP-O)4.  [Ref.  3]  STRAP-0  assesses  the  feasibility 
of  achieving  Navy  officer  manpower  plans.  Two  models  form  the  core  of  the  STRAP- 
O  system:  the  Navy  Officer  Force  Projection  (OPRO)  model  [Ref.  4].  and  the  Officer 
Retention  Forecasting  model  (ORFM)  [Ref.  5]. 

OPRO  uses  predictions  of  personnel  flow  rates  to  project  a  begin  year  inventory 
into  the  future  (by  year).  A  starting  inventory  is  successively  projected  (by  year)  into 
the  future.  This  allows  a  manager  to  assess  the  feasibility  of  manpower  goals,  test  the 
sensitivity  of  the  force  to  policy  changes,  and  develop  promotion  and  accession  plans. 

The  officer  retention  forecasting  model  (ORFM)  is  an  integrated  set  of  time- 
series  and  econometric  models  that  produce  loss  rate  forecasts  for  STRAP-O.  Loss  rate 
forecasts  are  generated  over  a  seven-year  horizon.  The  manager  has  the  capability  to 
alter  these  forecasts  through  a  change  in  the  real  value  of  military  pay  or  through  the 
selection  of  the  forecasting  technique. 


3The  complement  of  the  BFR,  1-BFR,  is  that  portion  of  the  CFB  who  resign 

their  commissions  annually. 

4 A  similar  system  for  Navy  enlisted  personnel,  the  Structured  Accession  Planning 
System-Enlisted  (STRAP-E),  is  also  operational  (see  Silverman,  1979). 
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STRAP-0  begins  by  estimating  loss  rates  using  ORFM,  and  then  applying  them 
to  current  inventories  in  OPRO.  Some  accession  and  promotion  policies  are  specified 
along  with  pay  grade  end  strength  targets.  OPRO  then  generates  inventories  and 
continuation  rates  describing  officer  personnel  flow  behavior  under  these  policy 
specifications. 

An  initial  Unrestricted  Line  (URL)  version  of  STRAP-O  was  installed  in  the 
Deputy  Chief  of  Naval  Operations  (OP-130)  in  September  1981.  A  total  officer  force 
version  became  operational  in  March  19S2. 

In  August  of  1986,  NPRDC  concluded  a  verification/validation  study.  FY1985 
loss  rates,  gains,  and  promotion  policies  were  used  to  project  FY85  end  strength  by 
community,  grade,  and  years  of  service  (YOS). 

NPRDC's  general  conclusions  were: 

1.  The  one-year  validation  of  losses  showed  that  no  one  methodology  excelled.  All 
four  methodologies  (Naive,  Weighted,  ACOL5,  and  MAD6)  underestimated 
losses  for  FY85. 

2.  The  two  error  measures  for  losses,  Weighted  Absolute  Error  (WAE)  and 
Weighted  Mean  Square  Error  (WMSE),  gave  nearly  identical  rankings  of  loss 
forecasting  capabilities. 

3.  The  one-year  validation  of  inventories  for  FY85  showed  that  no  one 
methodology  excelled.  Error  rates  tended  to  be  less  than  five  percent. 

Improving  inputs  to  the  forecasting  models  requires  accurate  estimates  of 
expected  losses.  Carol  Mullins,  from  NPRDC,  conducted  a  study  in  March  of  1986. 
Her  study  was  the  Development  of  a  Navy  Officer  End-of-Active-Obligated-Service 
(EAOS)  Date7. 

At  present,  the  minimum  service  requirement  (MSR)  date  is  the  only  measure  of 
service  obligation  available  on  an  officer's  computerized  personnel  record.  However, 
the  MSR  includes  only  obligation  incurred  by  source  of  entry  (e.g.  Naval  Academy) 
and  primary  warfare  training  (e.g.,  basic  flight  training).  Obligations  incurred  after  the 


ACOL  is  defined  as  the  present  value  of  the  monetary  returns  from  remaining  in 
the  Navy  for  one  more  period  and  then  making  the  optimal  stay  or  leave  decision, 
minus  the  present  value  of  the  monetary  returns  from  leaving  the  Navy  immediately. 

6MAD  is  defined  as  a  technique  that  minimizes  the  sum  of  the  absolute  values  of 
the  errors  between  historical  loss  rates  and  loss  rate  estimates. 

'The  EAOS  date  will  depend  on  whether  the  additional  obligation  was 
consecutive  or  concurrent.  In  a  special  version  of  the  concurrent  case,  the  additional 
obligation  is  exhausted  prior  to  completion  of  the  MSR.  Then  the  EAOS  date  is 
simply  the  MSR. 
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expiration  of  the  MSR  do  not  appear  on  the  personnel  record.  Because  use  of  the 
MSR  alone  can  lead  to  an  inaccurate  measurement  of  eligibility  (Bres  &  Rowe,  1983), 
a  more  comprehensive  measure  is  needed  to  capture  all  obligations.  The  study  is  a 
separation  of  officer  personnel  into  those  eligible  to  leave  the  Navy  and  those  still 
under  a  service  obligation.    [Rcf.  6:  Mullins,  1986,  page  1] 

The  two-stage  algorithm  extracts  any  data  elements  from  officer  master  file 
records  that  may  indicate  that  an  obligation  has  been  incurred.  The  EAOS  date  is 
computed  in  the  second  stage.  The  algorithm  accounts  for  changing  obligations  and 
types  across  time,  as  well  as  newly  implemented  obligation  programs. 

ACOL  and  MAD  techniques  are  primarily  used  in  the  STRAP-0  forecasting 
model.  With  the  advent  of  EAOS  and  defining  eligible  officers  more  accurately,  we 
could  conceivably  generate  a  more  accurate  loss  forecast.  This  would  be  incorporated 
in  both  ACOL  and  MAD  models. 

Because  the  present  forecasting  technique  combines  voluntary  and  involuntary 
losses  together  without  regard  to  who  is  actually  eligible  to  make  a  decision,  I  believe 
that  this  is  a  major  flaw.  If  one  knows  who  is  eligible  to  make  a  decision  his  forecast 
for  the  future  will  become  more  accurate.  Utilizing  EAOS  data  will  allow  us  to  improve 
the  forecast  and  I  will  attempt  to  prove  this  theory. 

The  next  chapter  will  explain  the  methodology  of  the  target  population,  data 
collected  and  variables  used.  In  chapter  three  the  analytical  materials  are  applied  to  the 
equations.  Chapter  Four  will  define  the  results  from  our  loss  equations  and  prove 
which  equation  is  actually  better.  Conclusions  for  manpower  losses  are  offered  in  the 
final  chapter. 
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II.  METHODOLOGY 

In  this  section,  the  data  used  to  analyze  the  forecasting  value  of  eligibility  data 
are  described. 

The  initial  investigation  was  limited  to  active  duty,  unrestricted  line,  pilots  from 
the  grade  of  Ensign  (0-1)  through  Lieutenant  Commander  (0-4).  The  management  of 
non-active  duty  officers  (non-active  duty  reservists  and  retirees)  is  sufficiently  different 
to  be  excluded  from  this  study.  Restricted  officers  (limited  duty  officers  (LDOs))  have  a 
unique  career  path  and  service  characteristics,  and  were  not  addressed  in  this  analysis. 

A.  DATA  COLLECTED 

The  data  used  in  this  thesis  were  derived  from  the  Officer  Personnel  Information 
System  (OPIS)  and  NPRDC's  officer  data  processing  system,  FAIM-O.  0P1S  is  an 
interactive  information  delivery  system.  Using  data  retrieval  and  display  software,  as 
well  as  data  organization  techniques,  the  system  provides  rapid  access  to  a  substantial 
amount  of  historical  officer  personnel  information.  OPIS  is  currently  housed  on  an 
IBM  43S1  mainframe  computer  at  NPRDC,  San  Diego.  Communication  to  NPRDC 
was  via  telephone  line.  The  Officer  Personnel  Information  System  (OPIS)  is  organized 
hierarchically  as  illustrated  in  Figure  2.1  . 

The  OPIS  database  consists  of  officer  personnel  inventories  and  flows  (e.g., 
losses,  promotions)  by  designator,  grade,  YOS,  sex,  ethnic  group,  source  of  entry,  and 
time  remaining  on  obligation  during  the  period  FY75-87.  All  data  except  MSR-based 
retention,  are  produced  by  the  FAIM-0  database.  FAIM-0  is  a  longitudinal  database 
built  and  maintained  by  NPRDC  for  OP- 130.  MSR-based  retention  data  were 
provided  by  OP-136D.    [Ref.  7] 

B.  DESCRIPTION  OF  DATA 

Historical  inventory  and  loss  data  were  collected  from  OPIS  for  FY75  through 
FY86.  Each  inventory  or  loss  variable  was  arrayed  by  pilot  community  (e.g.,  jet,  prop, 
helo),  paygradc,  and  years  of  completed  service. 

Navy  pilots  are  trained  and  managed  in  three  distinct  communities:  jet,  propellor 
("prop"),  and  helicopter  ("helo").  The  communitites  are  also  distinguished  by  their 
career  retention  behavior  (as  the  chart  shows).  While  I  originally  looked  at  officers  in 
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paygrades  0-1  to  0-4,  I  focused  on  0-3's.  They  represent  paygrade  from  which  most 
officers  leave  the  Navy  and  where  loss  rates  are  most  volatile  and  therefore  difficult  to 
predict.    Finally,    years    of    completed    service    (YCS)    dimension    is    measured    by 
subtracting  an  officer's  year  group  from  the  fiscal  year  in  which  the  officer  is  serving. 
The  following  inventory  and  loss  variables  were  used  in  the  study. 

1.  Inventory 

The  on-board  count  of  pilots  at  the  beginning  of  a  fiscal  year. 

2.  Eligibles 

The  eligibles  category  represents  the  number  of  pilots  with  between  zero  and 
less  than  one  year  of  service  remaining  on  their  contract  at  the  beginning  of  a  fiscal 
year.  This  group  is  frequently  referred  to  as  the  "population  at  risk".  The  "eligibility 
rate"  represents  that  portion  of  an  inventory  that  is  eligible  to  make  a  stay/leave 
decision. 

3.  Voluntary  Losses 

The  voluntary  losses  category  accounts  for  the  number  of  officers,  who  were 
eligible  to  stay  or  leave,  made  a  voluntary  decision  to  leave.  For  example,  a  voluntary 
loss  occurs  when  an  officer  resigns  or  retires. 

4.  Involuntary-Eligible  Losses 

The  involuntary-eligible  losses  category  includes  individuals  who,  while  eligible 
to  make  a  stay  or  leave  decision,  were  separated  from  the  service  involuntarily. 
Involuntary  losses  occur  through  death,  discharge,  or,  in  the  case  of  reservists,  release 
from  active  duty. 

5.  Ineligibles 

The  ineligibles  are  the  complement  of  eligibles  or  the  difference  between  the 
inventory  of  pilots  and  the  eligible  population.  The  ineligibles  category  represents 
pilots  who  had  more  than  one  year  of  service  remaining  on  their  obligations. 

6.  Involuntary-Ineligible  Losses 

The  involuntary-ineligible  category  losses  represents  ineligible  individuals. 
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III.  ANALYSIS  OF  DATA 

A.       CURRENT  METHODOLOGY 

None  of  the  existing  loss  forecasting  methods  used  by  the  Navy  incorporate 
knowledge  of  the  size  or  composition  of  the  current  or  expected  population  at  risk. 
Instead,  losses  are  forecasted  from  a  begin-year  inventory  by  first  forecasting  voluntary 
losses  from  the  inventory,  then  forecasting  involuntary  losses  from  the  same 
population,  and  finally  summing  the  two  forecasts.  The  equation  is: 

(1)  TOTAL  LOSSES/INV  =  VOL  LOSSES/INV  +  INVOL  LOSSES/INV 


where: 


INV 

VOL  LOSSES 

INVOL  LOSSES 


=  Begin  Fiscal  Year  Inventory 

=  Voluntary  Losses 
(e.g.,  resignations) 

=  Involuntary  Losses 
(e.g.,  discharge,  death) 


Viewed  another  way: 


Begin  Fiscal  Year  Inventory 

Continue 

Losses 

Voluntary 

Involuntary 

This  approach  assumes  implicitly  that  any  member  of  the  begin-inventory  can 
voluntarily  leave  the  Navy.  The  prediction  of  voluntary  losses  is  not  based  on 
knowledge  of  the  number  of  officers  that  can  actually  leave  voluntarily.  Conceivably, 
the  Navy  could  predict  more  voluntary  losses  from  a  begin  inventory  than  there  are 
officers  eligible  to  leave  from  the  inventory.  Stated  differently,  because  the  inventory  is 
not  separated  into  eligibles  and  incligiblcs,  analysis  of  policies  that  impact  eligibles  only 
(e.g.,  continuation  bonuses)  is  difficult. 
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B.       NEW  METHODOLOGY 

The  development  of  a  Navy  officer  EAOS  date  (Mullins,  1986)  provided  the  first 
opportunity  to  incorporate  eligibility  data  into  Navy  officer  loss  forecasting.  The 
revised  loss  forecasting  equation  becomes: 

(2)  TOTAL  LOSSES/INV  -  a(VOL  LOSSES/ELIG  +  INVOL  LOSSES/ELIG}  + 

b{INVOL/INELIG} 


where: 


=  Begin  Fiscal  Year  Inventory 

=  Voluntary  Losses 

=  Involuntary  Losses 

=  Number  of  Eligibles  in  Inventory 

=  Number  of  Ineligibles  in  Inventory 

=  fraction  of  Inventory  in 
forecast  year  that  is  eligible 

b  =  (1-a),  fraction  of  inventory  in 

forecast  year  that  is  ineligible 

Incorporating  the  eligibility  data  provides  a  different  conceptual  view: 


INV 

VOL  LOSSES 

INVOL  LOSSES 

ELIG 

INELIG 

a 


Begin  Fiscal  Year  Inventory 

Elieibles 
("Population  at  Risk") 

Ineligibles 

Continue 

Loss 

Continue 

Loss 

1 
Vol          Invol 

Invol 

In  addition  to  more  accurately  identifying  those  portions  of  a  begin-inventory 
capable  of  becoming  various  types  of  losses,  the  new  method  also  has  the  advantage  of 
using  knowledge  of  the  forecast  year  not  available  to  the  current  method.  Specifically, 
the  new  method  uses  the  eligible  and  ineligible  fractions  for  the  first  forecast  year.  For 
that  year  only,  the  rates  are  known  with  reasonable  certainty.  By  weighting  each  of 
the  sub-forecasts  by  their  respective  eligibility  fractions,  the  new  method  exploits  its 
knowledge  of  historical  loss  behavior  but,  tempers  it  with  its  knowledge  of  the  future. 
This  contrasts  to  the  current  method  which  depends  solely  on  historical  data  to 
generate  its  forecasts. 
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Figure  3.1,  suggests  that  knowledge  of  eligibility  might  be  important  to  loss 
forecasting.  The  eligibility  fraction  not  only  differs  among  jet,  prop,  and  hclo  pilots, 
but  has  changed  dramatically  over  the  last  12  years.  The  sharp  increase  in  eligibility  in 
the  early  80's  is  due  largely  to  the  introduction  of  the  Aviation  Officer  Continuation 

Pay  or  "pilot  bonus". 

C.       METHODS  OF  FORECASTING 

Since  the  primary  purpose  of  this  study  was  to  determine  the  impact  of  eligibility 
data  or  loss  forecasting,  the  study  used  simple  rate  forecasting  methods  to  generate  the 
sub-forecasts  in  equations  (1)  and  (2).  (In  practice,  the  Navy  employs  more 
sophisticated  methods  of  producing  the  sub-forecasts.)  Three  simple  extrapolation 
methods  were  chosen. 

1.  Naive  (PY) 

This  technique  suggests  that  next  year's  loss  rate  will  be  the  same  as  the 
current  year's  rate.  Mathematically,  it  is  stated  as: 


xt+i  ■  xt 


where  X*  is  the  loss  rate  for  year  t  ,  and  Xt+  j  is  the  estimated  loss  rate  for  year  t+  j 
Generally,  the  naive  model  is  accurate  only  if  the  data  displays  a  constant  trend. 
2.  Simple  (Unweighted)  Moving  Averages  (SM) 

This  approach  takes  a  simple  or  unweighted  moving  average  of  historical  loss 

rates  to  produce  a  forecast.  I  elected  to  use  a  three-year  moving  average.  The  equation 

is: 

3 

t  +  1       .     .       t  -  i  +  1 
i  =  1    ^ — 


3.  Weighted  Moving  Average  (VVM) 

If  trends  in  loss  behavior  suggest  that  the  forecast  should  resemble  the  recent 
past  more  than  the  distant  past,  then  a  weighted  moving  average  should  be  used.  The 
WM  method  uses  three  years  of  historical  data,  but  uses  a  3-2-1  descending  weighting 
scheme.  The  equation  is:  -j 

Xt  +  1    =     .E,       Xt-Ul    *    (3-U1) 
1=1  6 
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Figure  3.1     Loss  Eligibility. 
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IV.  RESULTS 

A.  PREDICTED  VERSUS  ACTUAL  ANALYSIS 

By  comparing  predicted  loss  rates  to  actual  loss  rates  in  Tables  1  ,  2.  and  3,  the 
first  conclusion  that  can  be  drawn  is  that  EAOS  forecasts  contribute  more  to  loss 
accuracy  than  does  the  BI  results. 

From  the  90  cases  analyzed,  (13  cases  were  discarded  because  of  "ties";  seventy- 
seven  cases  with  unambiguous  results  remained).  EAOS  predicted  46  cases  more 
accurately  than  BI.  BI  methods  were  more  accurate  in  31  cases.  For  example, 
FY  1982.  YCS  5,  jet  community,  the  actual  loss  rate  for  that  year  was  17  percent  or  49 
individuals.  The  best  predictor  for  that  time  period  was  EAOS  previous  year  (PY)  with 
a  19.2  percent  loss  rate  or  55  pilots.  Six  more  pilots  were  predicted  to  leave  than 
actually  left,  but  five  less  than  the  best  case  for  the  BI  technique.  Overall,  EAOS 
under-predicted  slightly  more  often  than  it  over-predicted.  EAOS  had  24  under- 
predictions  (52  percent),  19  over-predictions  (41  percent),  and  three  cases  where  actual 
equal  predicted  (7  percent).  In  contrast,  BI  methods  over-predicted  more  often.  BI 
had  19  over  (61  percent),  11  under  (36  percent),  and  one  equal  to  the  actual  (3 
percent). 

B.  WEIGHTED  AVERAGE  ERROR  OVER  TIME 

Table  4,  summarizes  the  validity  of  the  various  methods  over  time  (FY82-86),  but 
displaying  weighted  average  errors.  Each  year's  forecast  error  is  weighted  by  its  begin- 
inventory.  The  table  is  separated  into  YCS  and  aviation  community.  Fiscal  years 
82-86  are  again  represented. 

The  smallest  weighted  error  suggest  the  most  accurate  technique  over  the  fiscal 
year  validation  period.  For  example,  YCS  4,  jet  community  the  smallest  error  is  .004 
under  the  EAOS  WM  column.  In  YCS  five,  jet  community  the  smallest  error  is  .029 
under  the  EAOS  PY  column  and  so  on. 

In  the  jet  community  four  years  out  of  six  (67  percent),  BI  was  a  better  predictor. 
EAOS  predicted  better  in  the  prop  and  helo  communities  with  four  out  of  six  YCS  cells 
(67  percent).  Seven  out  of  eighteen  best  forecasting  cases  resulted  in  a  weighted  error 
of  .05  or  higher,  four  were  BI  and  three  EAOS.  EAOS  dominated  in  all  communities 
for  YCS  4  and  5.    Results  in  YCS  7  to  9  were  mixed. 
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TABLE  2 
PREDICTED  VERSUS  ACTUAL  LOSS  RATES  (COM) 
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TABLE  3 
PREDICTED  VERSUS  ACTUAL  LOSS  RATES  SUMMARY 
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TABLE  4 
WEIGHTED  AVERAGE  ERROR 
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TABLE  5 
TURNING  POINT  ANALYSIS 

PROP  5  YOS 

FY  BEST    OCT  LOSS  PRED  DEV  PLT  BEST  PRED  DEV 

81-82  EPOS  PY    23.6  25.6  +2.0  81  PY  27.0  +  3.4 

83-84  BI  PY      9.1  11.0  +1.9  EPOS  PY  12.2  +3.1 

HELQ  5  YOS 

FY  BEST    OCT  LOSS  PRED  DEV  PLT  BEST  PRED  DEV 

81-82  EPOS  PY     2.6  4.8  +2.2  BI  PY  6.0  +3.4 

82-83  EROS  SM     4.4  4.5  +0.1  BI  WM  5.3  +0.9 

JET  6  YOS 

FY  BEST    PCT  LOSS  PRED  DEV  PLT  BEST  PRED  DEV 

82-83  BI  WM     19.0  19.5  +0.5  EPOS  PY  14.0  -5.0 

83-84  BI  WM     15.1  18.8  +3.7  EPOS  SM  24.5  +3.4 

84-85  EPOS  SM    33.9  22.0  -11.9  BI  SM  17.3  -16.6 

PROP  6  YOS 

FY  BEST    PCT  LOSS  PRED  DEV  PLT  BEST  PRED  DEV 

83-84  EPOS  PY    20.5  21.3  +0.8  BI  SM  18.0  -2.5 

84-85  EPOS  SM    40.1  20.0  -20.1  BI  PY  20.0  -20.1 

JET  7  YOS 

FY  BEST    PCT  LOSS  PRED  DEV  PLT  BEST  PRED  DEV 

82-63  BI  SM     14.6  14.0  -0.6  EPOS  SM  6.7  -7.9 

83-84  BI  SM     12.6  12.3  -0.3  EROS  SM  13.6  +1.0 

84-85  EROS  SM    24.3  13.5  -10.8  BI  WM  12.5  -11.8 

PROP  7  YOS 

FY  BEST    PCT  LOSS  PRED  DEV  PLT  BEST  PRED  DEV 

81-82  BI  WM     13.2  14.3  +1.1  EROS  SM  9.1  -4.1 

82-83  EROS  PY     7.2  7.2  0  BI  WM  12.8  +5.6 

84-85  BI  PY     25.9  17.0  -8.3  EPOS  PY  14.7  -11.2 

HELO  7  YOS 

FY  BEST    PCT  LOSS  PRED  DEV  RLT  BEST  PRED  DEV 

84-85  BI  PY     14.1  10.0  -4.1  EROS  PY  5.3  -8.8 

JET  8  YOS 

FY  BEST    PCT  LOSS  PRED  DEV  RLT  BEST  PRED  DEV 

82-83  EPOS  PY     6.6  6.0  -0.6  BI  WM  3.8  +3.2 

83-84  BI  WM      7.8  8.2  +0.4  EROS  PY  8.4  +  0.  6 

84-85  EROS  SM    11.7  10.8  -0.9  BI  SM  8.3  -3.4 

PROP  8  YOS 

FY  BEST    PCT  LOSS  PRED  DEV  PLT  BEST  PRED  DEV 

82-83  EPOS  PY     5.6  4.6  -1.0  BI  PY  8.0  +2.4 

83-84  BI  PY      3.2  5.0  +1.8  EPOS  SM  5.1  +1.3 

84-85  EROS  SM    10.3  6.8  -4.1  BI  SM  5.3  -5.6 

HFLY°  BEYS?5   RCT  LOSS  PRED  ___^__^[^l__J^ ----- 

81-82  EROS  SM     1.0  *•  B  J-*  Efi0S5M  3.  7  -2.2 

e3_63  BI  SM      5.9  6-0  _■  wM  --7  _1b5 

84-65  EPOS  SM     5. 2  4. _ 

PREDICT 

>5  TURN  POINTS       OVER  UNDER  EVEN 

EPOS  14          5  J 

BI  11          7  * 
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C.      TURNING-POINT  ANALYSIS 

With  a  stable  flow  of  personnel,  it  is  relatively  easy  to  predict  loss  behavior. 
Most  techniques  should  predict  well  under  these  conditions.  However,  strong 
techniques  should  also  be  able  to  predict  changes  in  the  direction  of  rates  over  time. 
Statistically,  these  shifts  are  known  as  "turning  points". 

The  most  significant  turning  points  in  the  historical  data  are  listed  on  Table  5  . 
The  table  lists  the  significant  turning  point  year  and  which  method  was  the  better 
predictor.  It  then  lists  the  actual  loss  rate,  the  best  predicted  rate,  and  the  deviation 
under  or  over  from  the  actual  rate.  The  alternate  (second  best)  predictor  is  listed  along 
side  with  the  same  indicators  as  before.  As  an  example,  a  prop  pilot  with  YOS  5  had  a 
significant  turning  point  at  FY81-82.  In  1982  the  best  predictor  was  EAOS  with  a  rate 
of  25.6  percent,  the  actual  was  23.6  percent.  The  predicted  value  was  2.0  percent  over 
the  actual  rate.  The  alternate  would,  of  course,  be  BI  with  a  predicted  value  of  27.0 
percent  and  it  over  predicted  by  3.4  percent. 

Of  the  25  major  turning  points  selected  the  better  predictor  was  EAOS  with  14 
(56  percent)  and  BI  drew  11  (44  percent).  There  did  not  seem  to  be  a  major  difference 
between  the  two  methods.  When  tabulating  the  numbers  it  did  show  that  EAOS  under 
predicted  more  often,  while  BI  over  predicted  more  frequently. 
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V.  CONCLUSIONS 

This  study  suggests  that  by  utilizing  the  EAOS  date  loss  forecasting  techniques 
can  be  more  accurate.   The  following  findings  arc  germane: 

1.  Predicted  versus  actual  loss  rates  showed  that  EAOS  would  perform  better  than 
the  present  system  described  by  the  BI  method.  EAOS  predicted  better  60 
percent  of  the  time. 

2.  Over  a  five  year  validation  horizon,  EAOS  predicted  better  56  percent  of  the 
time.  BI  was  better  in  the  jet  community,  but  EAOS  performed  better  in  the 
prop  and  helo  community. 

3.  Turning  point  analysis  is  another  important  criteria  for  evaluating  loss  rate 
forecasting  techniques.  EAOS  again  predicted  56  percent,  whereas  BI  was  44 
percent. 

This  analysis  suggests  that  the  EAOS  data  will  be  absorbed  into  the  present 

forecasting  techniques  through  MAD  and  ACOL  models  incorporated  within  STRAP- 

O. 
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